fix(sqlite_writer): skip index cells that exceed a single page#303
Merged
DeusData merged 1 commit intoDeusData:mainfrom May 10, 2026
Merged
fix(sqlite_writer): skip index cells that exceed a single page#303DeusData merged 1 commit intoDeusData:mainfrom
DeusData merged 1 commit intoDeusData:mainfrom
Conversation
write_index_btree() flushed the page buffer whenever the next cell didn't fit, on the assumption that a freshly-initialised page would always accept any cell. That invariant fails for index cells whose payload (qualified_name, file_path, etc.) is larger than a full SQLite page — after the flush, pb_cell_fits() is still false but the code calls pb_add_cell() anyway. The subsequent content_offset underflow corrupts the page header and the next write triggers SIGBUS on large repos (observed on a Ruby codebase with ~215K nodes). Re-check pb_cell_fits() after the flush and continue past cells that still don't fit. Index entries whose keys exceed a full page can't be stored and are not expected to survive the writer anyway — the rest of the index is correctly preserved and the indexer no longer crashes. Record overflow pages (DeusData#175) already covers the record-btree side of this same shape of bug.
DeusData
added
bug
Owner
|
Merged via rebase, thanks @jjoos. The diagnosis is dead on — the asymmetry between the record-btree path (overflow pages, fixed in #175) and the index-btree path (no overflow mechanism) is exactly why the same shape of guard produced two different bug surfaces, and the fix preserves SQLite's invariants without inventing a new overflow scheme. Following up on |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
write_index_btree()crashes with SIGBUS when indexing large repositories (observed on a Ruby codebase with ~215K nodes, specifically the HackerOnecoremonolith with ~67K functions).The crash occurs in the page-buffer writer when an index cell's payload (qualified_name + file_path + properties) exceeds the SQLite page size. The code flushes the current page and assumes the fresh page will always accept the cell — but oversized cells still fail
pb_cell_fits()after the flush. The subsequentpb_add_cell()causes acontent_offsetunderflow, corrupting the page header and triggering SIGBUS on the next write.Reproduction
Index any repository with functions whose qualified names + file paths exceed ~4000 bytes combined (the SQLite page size minus overhead). A Ruby monolith with deeply nested namespaces and long file paths reliably triggers this.
codebase-memory-mcp index --path /path/to/large-ruby-repo # → SIGBUS in write_index_btree at sqlite_writer.c:1218Fix
After
pb_promote_and_flush(), re-checkpb_cell_fits(). If the cell still doesn't fit on an empty page,continuepast it. Index entries whose keys exceed a full page cannot be stored in a leaf and are silently dropped — the rest of the index is correctly preserved.This is the index-btree counterpart to PR #175 (record overflow pages), which fixed the same shape of bug for the record-btree writer. The record side uses overflow pages; the index side has no overflow mechanism, so skipping is the correct behavior.
Changes
internal/cbm/sqlite_writer.c: 10 lines added, 3 removed inwrite_index_btree().Testing
Related